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ABSTRACT 

The overall purpose of these papers is to provide a 
background working paper to serve as a basis for developing a system 
for evaluating states’ programs in Career Guidance, Counseling and 
Placement. An extensive review of published materials is summarized 
into three areas: (1) the accomplishments, limitations and trends of 

the current guidance effort: (2) the major concerns of guidance 

people; and (3) criteria for evaluating programs and identifying 
methods of improving vocational guidance services. Taking this 
information as a point of departure, the second paper recommends a 
procedure for securing the information, within a state, about major 
areas of concern and about evaluative criteria, i.e. evidences which 
show a program to be effective. A samplinq of pertinent state level 
personnel as well as local administrators, student, teachers, and 
teacher* and counselor-educators were presented lists of identified 
items under each classification and asked to rank them. Results were 
presented. (TL) 
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' EVALUATING CAREER GUIDANCE, COUNSELING, 
AND PLACEMENT- -STATE LEVEL 



By 

-Dr. William W. Stevenson, Assistant State Director; 
and, Head, Division of Research, Planning, and Evaluation 
Oklahoma State Department of Vocational and Technical Education 

and 

Mr. Blan E. Sandlin, Administrator 
Guidance and Counseling Division 
Oklahoma State Department of Education 



Introduction 



This paper is intended to be the second part of a three -part presentation 
dealing with evaluating career guidance, counseling, and placement at (1) the 
national level, (2) the state level, and (3) the local level. While it is 
difficult in many instances to separate the efforts and materials relating to 
guidance and counseling at these three levels, this paper will attempt to deal 
with the problem at the state level. We should also point out that we are 
speaking of the three areas mentioned in the topic - career guidance, counsel- 
ing, and placement. It may be that at cimes we will not use all three of the 
terms, but we do view the functions of the total efforts ms including all three 
of these areas. 

This presentation will }$ divided into two parts. The first part of the 
presentation will concentrate upon the accomplishments, limitations and trends 
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In the guidance area as well as the major areas of concern and information 
needed for evaluation as identified by a review of the literature. The second 
part of the paper will present one method of identifying the major concerns 
and needed evidences within a state and the results of having applied this 
method in a particular state. 

This paper will present a review of published materials collected from a 
number of sources; primarily three areas will be treated--first, accomplishment , 
limitation, and trends of the guidance effort; second, a synthesis of the areas 
of concern; and third, the kinds of evidence that have been asked for or other- 
wise should be provided in order to evaluate programs and identify methods of 
improvement of career guidance, counseling, and placement. Each of these three 
areas will be covered in more detail as we move through the presentation. We 
hope to present to you some suggestions which may be further refined or modified 
in your own conmittee work. We will offer our conclusions as to what the major 
concerns are in this field and suggest some procedures whereby the programs may 
be evaluated at the state level. We also would hope to bring to your attention 
some of the accomplishments, limitations, and trends, in the areas of career 
guidance, counseling, and placement. We do wish to stress the fact that the 
paper which is being presented is Intended to be only a beginning statement of 
the situation and we do want to emphasise the fact that we are perfectly amenable 
to modification and Improvement. 

Statement of the Problem 

The theme of the Ouldance Divlaion Program--the title of thla presentation-- 
ls a challenge within itself. I suspect that some people in this audience and 
many of the guidance people I have talked with do not agree that the three 
areas mentioned in the title should be or are the central concern of guidance 
and counseling programs. ft>e use of the term career guidance, particularly if 



it relates to a career which may be based only on a high school level education, 
would in many circles be questionable. Probably, the most controversial part 
of the topic would be the inclusion of the term placement. I have found many 
people who would disagree with the statement that placement should be a part 
cf the guidance and counseling service of the state. I have found very little 
effort being expended in the placement of students by those working in the guid- 
ance area. Thus we conclude that placement may not in many instances be an 
accepted function of people working in this area. 

Let me quote briefly from a few statements made relative to this particular 
question. Dr. Po-yen Koo, Director of Data Collection and Evaluation, in the 
New Jersey State Department of Education has this to say under a topic entitled, 
"Vocational Guidance: Where are You?" 

A number of studies have been made of the means used by voca- 
tional program graduates to obtain their first employment. These 
include nation-wide studies (Eninger, 1968, ED012315)> regional 
studies (Kaufman et al., 1967 , ED011060), and local studies. 

The preponderance of the evidence in these reports indicates 
thai fewer than one-third of vocational program graduates obtained 
first employment through school placement. Rather, the great 
majority of them rely heavily upon family, friends, and other 
channels to obtain a first job. 

Is there anything that the school can do to improve the 
situation? 

These findings should have some important bearing on the problem 
of vocational guidance, a primary function of the secondary school. 

New graduates from a vocational program not only require placement 
service, but may very well need assistance in adjustment, particularly 
in relation to their first job. If they have not had assistance 
from the school in getting a Job in the first place, there seems 
little possibility for them to seek and receive guidance from the 
school when facing a need for adjustment arising from difficulties 
in work. 

In Dover, Delaware, a newspaper article quotes Governor Russell W. Peterson 
in a meeting with vocational and guidance workers as asking the question, "What's 
wrong with schools being employment agencies?" Dr. Kenneth C. Madden who is 
superintendent of the Department of Public Instruction, replied that almost no 
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schools in the state provide this service to any great degree. Even more 
severe criticism may be interpreted in the findings of the study of 30,000 
high school seniors in Oklahoma in 1967 in which it was found that one -third 
of the seniors in high schools of Oklahoma said that counselors or teachers 
had had no influence on their career decisions, 5 4$ indicated some influence 
while only 12^ said teachers or counselors had influenced their decisions "Quite 
a Bit." The Oklahoma Advisory Council for vocational-technical education says 
that Oklahomans do not know about the program of vocational-technical education, 
are unaware of the intellectual and financial fulfillment of vocational and 
technical careers and are engaged in ever-increasing numbers in a tragic cycle 
of sending their youngsters to college to fail or drop out. In Occupational 
Information , by Robert Hoppock, we read the following paragraph: 

The.fi is fantasy in the belief that good vocational counseling 
can be provided by psychologists who know all about values and 
emotions and interests and capacities and who are not equally 
competent in the area of occupational information. 0 in 2 berg's 
hopeful statement that "...the vocational guidance counselors have 
always considered the reality factors in the environment of the 
individual" seems not to be universally true. Instead, what we 
now have in vocational counseling is far too many psychologists who 
regard placement as a dirty word and any direct contact with the 
employment market as degrading. If we are not someday to be charged 
with quackery, we should have in all vocational counseling services 
as many persons who are skilled in psychology, or we should have 
a new breed of counselors whose training, experience, and competence 
in economics, in occupational information, and in placement equals 
his training, experience, and competence in psychology, in psycho- 
metrics, and in psychotherapy . 

VOCATIONAL GUIDANCE. The classic definition of vocational 
guidance, formulated and revised by the National Vocational Guidance 
Association, reads as follows: "Vocational guidance is the process • 

of assisting the individual to choose an occupation, prepare for 
it, enter upon and progress in it." 

Enough of lamentations, incriminations and criticisms. He need to look 

< 

at the problem as a challenge for seeking a solution. Ihere has been too much 
of criticism and too little of worthwhile constructive suggestions for improve- 
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ment. 
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Objectives of the Paper 



The objective of this work is to provide a background working paper which 
will serve as a basis for the development of a continuing system for evaluation 
of the states' programs of Career Guidance, Counseling, and Placement. 

To reach this objective, we will review the accomplishment, limitations, 
trends, major concerns, and evidences needed for evaluation in rega'u to pro- 
grams of Career Guidance, Counseling, and Placement. A procedure for determin- 
ing acceptable evaluative criteria and the results of the application of that 
procedure will be presented. 

Finally, the authors will make certain recommendations for change based on 
the results of a review of the literature. 

Accomplishments 

The National Defense Education Act, Title V-A, provided the stimulus for 
the growth of guidance, counseling, and testing in the public schools of the 
nation. According to the Fifth Annual Report of the State Advisory Council on 
the State Departments of Education, there were, in 1958-59* 13*000 (full-time 
equivalent) counselors serving public school secondary students. Ry the end 
of the 1967-68 school year, the number had been increased to 1*0,450. In 1964, 
Title V-A was extended to include youth in elementary schools, Junior colleges, 
and technical inutitutes. During the 1967-68 school year, 6,689 counselors 
were serving these groups, making the total number somewhat more than 50,000 
(full-time equivalent) counselors. The counselor-student ratio in the elemen- 
tary school has been reduced since 1964, but not nearly as dramatically as at 
the secondary level. 

The total national expenditure for guidance end counseling after the first 
year of Title V-A from all sources was $10,833*000. Federal funds accounted 
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for 45 percent of the total. In 1968-69 school year, the total expenditure 
was more than $344 million, the Federal share being less than five percent. 

All state education agencies in cooperation with counselor education insti 
tutions have intensified efforts to improve the level of counselor qualifica- 
tions, Counselor certification plans hove been established in all states. 

There is much evidence that counselor certification and training is being up- 
graded in order to improve counselor competences in all areas of guidance and 
counseling. 

Most all states have established Guidance, Counseling and Testing Sections 
in the State Department of Kducation. Minimum qualifications for supervisors 
have been included within the State Plan. The State Plans also include minimum 
standards for certification of local school counselors, testing and guidance 
services necessary for an approved program, 

A well organized and conducted guidance program will use every resource 
and agency which cen contribute to the effectiveness end efficiency of the pro- 
gram. Many private agencies have, with the use of experts, produced and made 
available many valuable printed materials that are useful in all aspects of 
the guidance program. The Bureau of Employment Security, the Department of 
Labor and other government agencies both Federal and State have, through re- 
search and pilot surveys, provided up-to-date information about the labor 
market, Job openings and projected trends in employment that are useful tools 
for counselors who worked with students in career development. Some of the 
more useful government sponsored publications are, Dictionary of Occupational 
Titles , Handbook on Job Market Research Methods , Ares Skill Survey and Guides 
to. Local Occupational In f ormation , and The Occupational Outlook Handbook . 

Congress has recognized vocational guidance as an essential port of a 
number of operational programs for which funds have been authorized. The acts 
of Congress listed below Illustrate the concern of our Federal government. 



1. Public Law 73-30, the Wagner-Peyser Act of 1933. Establishment 
of the U. S. Employment Service. 

2. Public Law 78-16, the Disabled Veterans Rehabilitation Act of 
1943; Public Law 78-346, the GI Bill of 1944, and as amended in 
later years, the War Orphans EducaiLonal Assistance Act of 1961. 

3. Public Law 83-565, the Vocational Rehabilitation Act of 1954. 

4. Public Law 85-864, the National Defense Education Act of 1958, and 

subsequent amendments; Public Law 88-210, Public Law 88-665. 

5. Public Law 38-415, the Manpower Development and Training Act of 
1962, and amendments. (Area Redevelopment Act of 1961) 

6. Public Law 88-164, the Mental Retardation Facilities and Community 
Mental Health Centers Construction Act of 1963. 

7. Public Law 88-452, the Economic Opportunity Act of 1964. 

8. Public Law 89-IO, the Elementary and Secondary Education Act of 
1965. Local and State financial support. 

9. Public Law 88-210, the Vocational Education Act of 1963. 

10. Public Law 90-576, the amendments to the Vocational Education Act 

of 1963. 



Limitations 

With all of these accomplishments, there are still limitations to the full 
effectiveness of programs. Academic guidance services developed rapidly as a 
result of the National Defense Act. Almost nine out of every ten high schools 
in the United States provided academic counseling. In contrast, the occupa- 
tionally oriented youth got very little or no attention. Hie reasons for this 
inattention are many, some of which are: (a) Counselors for the most part come 

from the teaching ranks. Their education, training and experience were academic. 
Therefore, they were oriented toward the academic and knew very little about 
the world of work, (b) Most state counselor certification programs and coun- 
selor training institutions required only one course in occupations and voca- 
tions for counselor certification (a recent survey shows that most all states 
are now requiring only one three-hour course in this area), (c) Hie counselor- 
student ratio has been greatly reduced. However, it is still too high for the 
counselor to provide all of the services needed by all of the students, (d) Too 
many guidance programs have been crisis centered and remedial rather than pre- 
ventive and developmental, (e) Developmental vocational and occupational 



guidance programs have not been posjible because of the very limited number of 
comprehensive elementary school guidance programs, (f) The public has not been 
convinced to the point that it demands equal guidance services for the non- 
college bound student. 

When federal funds for guidance, counseling and testing are compared to 
funds provided for other educational programs we see that guidance has received 
only seed money. In 1966* only one half of the states were using vocational 
education funds to provide at least one person on the state staff to have 
responsibilities fcr vocational guidance and counseling, As late as 1966, less 
than one percent of vocational education funds were spent for vocational guidance. 

Resource material for organizing and conducting developmental vocational 
guidance programs K -12 is very limited. Uhls is also true in the area of career 
exploration. The curriculum in most local schools is so restricted to the 
academic requirements that students do not have, a chance to explore in the area 
of careers which could help them find a real reason for learning "reading, 
riting and rithmetic." Students have not been permitted to move into and out 
of vocational-technical programs and to select mixtures of vocational-technical 
and academic courses. 



Trends 



Significant events are taking place in and out of school today which vitally 
affect the Job of the counselor and teacher, and the services they provide. 
Important changes are occurring in both theory and practice. This section of 
the paper will enumerate the trends which become apparent as one reviews the 
literature. 

This probably is our way of being optimistic about the future, but it would 
appear that a massive effort is being made to change the thinking, the orienta- 
tion, and the outlook of career guidance personnel. If our reading of the 
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trends is correct, we see the possioility that guidance and counseling can 
become a part, a more effective part, an all , encompassing part of the total 
effort to properly train students for whatever may be their life's goals and 
ambitions. 

The trends as seen in our study of the literature are the following: 

1. Toward becoming a Career Development Specialist assisting students 
to formulate a career goal and develop plans for and adopt pro- 
cedures to reach that goal.' This trend has an opposite to it 
which is a trend away from being a crisis oriented therapist. 

2. To function in such a way as to be prepared to show evidences 
of accomplishments - Evaluation. It means that formulation of 
measurable objectives by guidance people. 

3. Consider both interest and aptitude of student in counseling 
concerning what is best for the student without prejudice on the 
part of the counselor. The counselor must to the extent possible 
avoid showing to the student in any way his evaluation of the 
student's goals and aims in life. 

4. Toward more contact with an involvement of the total community 
in the development and function of the guidance programs. 

Certainly possibly more of a goal than a trend, but a hoped for 
result. 

5. Starting career exploration and development work at an earlier 
age (elementary school) and going on as a developmental process 
throughout life. 

6. Career development accepted as a responsibility of the total 
school which cannot be limited to a single discipline or 
department. 

7. The trend toward accepting responsibility for placement of students 
and graduates on Jobs is in the talking stage at least. As said 
earlier, too little of it is being done. 

8. Toward more emphasis on providing services for the disadvantaged 
and handicapped. 



Areas of Major Concern 



What are the problems relative to career guidance, counseling, and place 
ment as seen by people at the various levels? I am sure that many of the con 
cerns identified at the state level will also be mentioned as concerns at the 
national and local level. 

Some of the problems that plague guidance, counseling, and placement pro 
grams as revealed by a search of the literature are: (1) Many schools do not 
have guidance counselors. Small poorly financed rural school districts are 
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more likely not to provide the services of a guidance counselor. (2) The 
counselor-student ratio in most schools is so high that counselors do not have 
time to provide the guidance services necessary for career development. This 
condition is especially true in the elementary school. (3) Counselors are 
often assigned duties that are in no way related to guidance. In fact, many 
duties assigned to counselors hinder their effectiveness to render guidance 
services. (4) Due to their training, education, experiences and lack of knowledge 
concerning the world of work, many counselors tend to over-emphasize the bacca- 
laureate degree, (5) Counselors are not involved in designing regular or 
remedial school programs that will meet individuals' needs based on interests 
and abilities. (6) Students, teachers, parents, and the general public are 
not adequately informed about the dignity, challenge (intellectual and financial) 
and satisfaction of vocational-technical skilled Jobs. (7) Counselors, teachers, 
and parents have misinformation concerning ability of students who take vocational- 
technical courses in high school to attend and succeed in college. (8) Students 
have not had a chance to gain experiences that will assist them in making wise 
educational and occupational choices. (9) Many schools do not have enough informa- 
tion available concerning the world of work for the guidance staff to develop a 
systematic program for acquiring knowledge about the world of work and for pack- 
aging and dissemination to the students. (10) Adequate communication lines do 
not exist between all agencies involved in all types of educational training and 
the various employment agencies. 

Other concerns relating to guidance, counseling and placement are: (ll) 

Guidance, counseling, and placement may not be an integral part of the total educa- 
tion program. (12) Vocational guidance and academic guidance might be performed 
by two different counselors, thereby further dividing the vocational and academic 
inclined students. (13) Local school counselors are not organizing and using 
advisory committees to assist in providing services needed by all the students 



in the community. (14) Research through follow-up studios of graduates and 
dropouts should be used to evaluate the effectiveness of career guidance activ- 
ities. (15) Establish procedures for implementing new ideas related to guidance, 
counseling, and placement. (16) Too many students are graduating or leaving 
school without a saleable skill. (17) Educational programs must be flexible and 
guidance programs responsive enough to design programs to meet individual needs, 
desires and abilities. (18) Too little effort is being expended to provide train 
ing and assistance for counselors in areas of occupational education. (19) Short 
supply of support personnel for counselors} i.e., test technicians, placement 
specialists, information specialises, etc. (20) The need for a broadly based 
guidance committee which will reflect the views of many sections of the education 
and ley public. 



Evidences Needed for Evaluation 



As we searched the literature, we found that the people were asking ques- 
tions, they were asking for information as to the value or the quality of the 
program of career guidance, counseling, and placement. 

The following is our interpretation of what data or information is being 
asked for or should be supplied in order to evaluate programs of career guidance, 
counseling, and placement. 

1. Is the guidance program influencing the school curriculum at all levels? 

2. Arc students making realistic educational and/or vocational plans 
based on predetermined Interests, test scores and/or grade point 
averages? 

3. What experiences have students had to help develop ability to make 
educational and/or vocational decisions? 

4. To what extent are post high school activities of graduates and 
school leavers consistent with their high school program? 
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5. Do high school graduates have either a job entry skill or a plan 
for continuing education? 

6. What percent of the vocational-technical student graduates get 
related jobs or continue in related training? 

7. Are students of all levels of ability participating in vocational- 
technical education programs? 

8. What percent of junior high school students have participated in 
career exploration and testing for work aptitude and interest? 

9. What percent of the students who drop out of college the first 
year take jobs related to vocational-technical training in high 
school or enter advanced training courses related to the high 
school courses? 

10. What percent of the counselors have available to them in-service 
workshops related to vocational aspects of guidance and what per- 
cent take advantage of the workshops that are available? 

11. What percent of the counselor's time is spent on activities 
other than guidance, counseling, and placement? 

12. Are materials for career guidance identified at the state level 
and made available or made known to local school counselors? 

13. Percent of counselor education dealing with occupations requiring 
vocational-technical training. 

14. Ratio of counselors to students. 

15. Percent of time spent by counselor on placement of graduates and/or 
school leavers. 

16. Changes in enrollment in vocational programs in school or area 

school. 

17. State Staff - Number of visits per school year. 

O 
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18. Length of time required from request to visit by state staff. 

19. Number of areas of expertise represented on state staff, l.e., 
elementary, testing, vocational, research, evaluation, adminis- 
tration, secondary, Information, publications. 

Recommendations 

We present the following recommendations for consideration: 

1. Professional preparation programs for guidance counselors should 
be modified to Include more training and Information related to 
the vocational-technical aspects of guidance. 

2. State Departments of Education should provide leadership In the 
promotion of local, regional and state seminars, workshops and 
conferences to assist counselors In becoming more proficient In 
career guidance, counseling and placement. 

3. State Departments of Education should provide the leadership 
necessary for promoting developmental career guidance for all 
children beginning In the first grade. 

4. An opportunity should be provided for all junior high school 
youth to engage In career exploration and to be tested for work 
aptitude In careers below the so-called professional level. 

5. Programs should be Initiated for providing opportunities and 
encouragement to practicing counselors at the high school level 
and post high school level to up-date business and industrial 
experiences In order to keep up with the ever-changing world of 
work. 

6. Greater emphasis should be placed on public information activities 
designed to Increase the awareness of students, their parents and 
other adults of the wide variety of challenging and rewarding 



occupational opportunities that exist In the world of work and 
the availability of training programs designed to prepare persons 
to enter these fields. 

Every school district should have a guidance committee made up 
of representatives from the various disciplines and other selected 
persons. One of the first duties of this committee would be to 
write a plan for guidance services. The plan should Include a 
job description for counselors that give high priority to com- 
petency In career guidance, counseling and placement. 

Guidance plans should have quantifiable objectives In order that 
more precise evaluation may be done. 

Guidance counselors In local schools should assist the school 
administrator In the establishment of an advisory council In which 
the majority of members are from Industry and business to lend 
emphasis to the vocational-technical aspects of guidance. 

Guidance Sections of State Departments of Education should provide 
leadership In the development of guides and brochures that set 
forth examples of units on career exploration and development to 
be used In establishing a developmental program K-12. 

Guidance Sections of the State Departments of Education should 
work cooperatively with local school counselors in developing 
more effective career guidance, counseling, and placement pro- 
gram for "special needs" youth (dropouts, handicapped, and dis- 
advantaged). 

State Guidance Sections, professional counselors organizations 
and Local and State Advisory Councils should present a program 
to the stele legislature for financing guidance services so that 



all schools can reduce the counselor-student ratio to the point 
that an effective guidance, counseling, and placement program 
for all students Is possible. 



Note: The second part of this report will be presented by Dr. William W. 

Stevenson, Head, Division of Research, Planning, and Evaluation} and Assistant 
State Director, Oklahoma State Department of Vocational and Technical Education. 



BIBLIOGRAPHY 



A Guide for the Development, Implementation and Administration of Exemplary 
Programs and Projects in Vocational Education, Atlanta, Georgia, 1969* 

A Guide for Developmental Vocational Guidance, Grades K-12, Oklahoma State 
Department of Education, 1968* 

An Administrative Guide for Evaluating Selected Aspects of High School Guidance 
Programs, Educational and Career Planning, California State Department of 
Education, 1966* 

Annual Report, National Advisory Council on Vocational Education, Vocational 
Education Amendments of 1968, Public Law 90-576, 1969* 

A Study of Counselor Education in California, California State Department of 
Education, 1967* 

Career Guidance, Counseling, and Placement Proceedings National Conference on 
Guidance, Counseling and Placement in Career Development and Educational- 
Occupational Decision Making, University of Missouri, 1969. 

Computer-Based Vocational Guidance Systems, U. S. Department of Health, Education, 
and Welfare, Office of Education, Washington, D. C., 1969* 

Education for A Changing World of Work, Report of the Panel of Consultants on 
Vocational Education, U. S. Department of Health, Education, and Welfare, 
Washington, D. C., 1963* 

Education for a Changing World of Work, Appendix III, The Economic and Social 
Background of Vocational Education in the United States, A Sociological Analysis 
of Vocational Education in the United States, The Case for Education for Home 
and Family Living, The Contribution to the National Economy of the Use of 
Resources Within and by the Family, Report of the Panel of Consultants on 
Vocational Education, Requested by the President of the United States, U. S. 
Department of Health, Education, and Welfare. 

First Report, Advisory Council to the Oklahoma State Board of Vocational and 
Technical Education, Vocational Education Amendments of 1968, Public Law 90-576, 

1969. 

Futures in Vocational Education, Michigan Department of Education, 1969* 

Guidance Programs and Their Impact on Students, Tamminen and Miller, 1968. 

Guidance Services, State Board for Vocational Education, Nevada, 1970. 

Guid-O-Rama, A Study of the Attitudes of Public Secondary School Counselors 
and Principals in Mississippi Toward Basic Guidance Services, Mississippi 
Department of Education, 1970. 

Highlights and Recommendations for Vocational Technical and Adult Education, 

West Virginia, 1968. 



O 

ERiC 



17 



BIBLIOGRAPHY, CON'T 



Information Handbook for Vocational Education In Missouri, State Department of 
Education, Vocational Division, Missouri, 1970. 

Man, Education and Work, Post-Secondary Vocational and Technical Education, 
American Council on Education, 1964-. 

Notes and Working Papers Concerning the Administration of Programs, Authorized 
Under Vocational Education Act of 1963 Public Law 88-210, As Amended, Prepared 
for the Subcommittee on Education of the Committee on Labor and Public Welfare, 
United States Senate, March, 1968. 

Preparing Evaluation Reports, A Guide for Authors, U. S. Department of Health, 
Education and Welfare . 

Public Law 88-210, 88th Congress, H.R. 4955> 19^3 • (Vocational Ed. Act 1963) 

Research Summary 1970-71, Marking and Reporting Pupil Progress, Research Division, 
National Education Association, 1970* 

Second Report of the National Advisory Council on Vocational Education. 

State Advisory Committee for Guidance and Counseling, Committee Report, Kansas 
State Department of Education, 1970. 

Summary, Survey of Vocational Guidance Activities in the Secondary Schools of 
Utah. 

The Vocational Guidance Quarterly, June, 1970. 

Third Report of the National Advisory Council on Vocational Education. 

Tucson Job Opportunities for Young People, Career Development and Occupational 
Information Institute, 1970. 

Vocational Education Amendements of 1968* 

Vocational Students' Perception of Guidance Needs, A Study Conducted in Minnesota 
Area Vocational-Technical Schools, Minnesota Department of Education, 1969. 

Vocational Education, The Bridge Between Man and His Work, General Report of the 
Advisory Council on Vocational Education, 1968. 

Vocational Education and Guidance, A System for the Seventies, James A Rhodes, 
Charles E. Merrill Publishing Co., 1970. 

WANTED: Active-Involved Counselors, Norman C. Gy3bers, Associate Professor of 

Education, University of Missouri, Columbia, Missouri. 

Working Together, Career Development and Occupational Information Institute, 
Arizona State University, 1969 . 



EDO 46009 



IDENTIFYING AREAS OF CONCERN AND DETERMINING EVIDENCES 
NEEDED FOR EVALUATION-CAREER GUIDANCE, COUNSELING, 
AND PLACEMENT-STATE LEVEL 



By 

Dr. William W. Stevenson, Assistant State Director; 
and. Head, Division of Research, Planning, and Evaluation 
Oklahoma State Department of Vocational and Technical Education 

and 

Mr. Blan E. Sandlin, Director 
Guidance and Counseling Division 
Oklahoma State Department of Education 



U S. DEPARTMENT Of HEALTH. EDUCATION 
4 WELFARE 
OFFICE OF EDUCATION 
THIS DOCUMENT MAS BEEN REPRODUCED 
EXACTLY AS RECEIVED FROM THE PERSON OR 
ORGANIZATION ORIGINATING IT POINTS OF 
VIEW OR OPINIONS STATEO DO NOT NECES- 
SARILY REPRESENT OFFICIAL OFFICE OF EDU- 
CATION POSITION OR POLICY 



Presented to the Guidance Division 
American Vocational Association Convention 
December 6, 1970 



tr\ 

vO 

o 

kD 

o 

o 



ierJc 



19 



IDENTIFYING AREAS OF CONCERN AND DETERMINING EVIDENCES 
NEEDED FOR EVALUATION-CAREER GUIDANCE, COUNSELING, 

AND PLACEMENT-STATE LEVEL 

By 

Dr. William W. Stevenson, Assistant State Director; 
and, Head, Division of Research, Planning, and Evaluation 
Oklahoma State Department of Vocational and Technical Education 

and 

Mr. 81 an E. Sandlin, Director 
Guidance and Counseling Division 
Oklahoma State Department of Education 

Introduction 

This paoer Is the last half of the presentation entitled, "Evaluating 
Career Guidance, Counseling, and Placement— State Level." The preceedlng 
report presented you with information and conclusions drawn from an 
extensive review of the literature. This portion of the presentation will 
focus your attention upon reconmended procedures for determining (1) the 
major concerns within a state relative to career guidance, counseling, and 
placement programs, and (2) the data and/or Information which would be 
acceptable for evaluating programs of career guidance and counseling. In 
addition to recommending a procedure for securing the above Information, 
we will present to you the results of having followed this procedure on a 
limited basis within a state. In other words, we will bring you a resume 
of the thinking of selected Individuals within a state as determined by the 
procedures recommended In this paper. 

Someone has said, "Faith, hope, and charity have characterized the 
American attitude toward guidance programs- -faith In their effectiveness, 



hope that they can meet Important If not always clearly specified needs, and 
charity In not demanding more evaluative evidence that the faith and hope 
are justified." Evaluation Is Indeed a difficult and time consuming task j 
however, no longer can we avoid doing an acceptable job of evaluation simply 
because of its difficulty. We must begin (as some states have) to perfect 
the process of making objective, reliable, and valid evaluations of all parts 
of the educational process Including career guidance, counseling, and 
placement. 

There are two types of evaluative criteria. I prefer to distinguish 
between these two types of evaluative information by labeling them (1) factors 
which show quality of programs, and (2) factors which cause quality of programs. 
This distinction may be the same as the distinction made when we talk of 
process evaluation and product evaluation. The "shows quality" and the 
"causes quality" terms are clearer for my understanding of the distinction 
between the two types of evaluative evidence. 

If we analyze the situation carefully, we should be able to identify 
a number of the factors which show quality of program. These factors taken 
from the list which was given by the earlier speaker would include such 
items as (1) percent of graduates and school leavers who have learned a 
saleable skill, (2) amount of remedial education required, (3) amount of 
information students, parents, and the general public have about the world 
of work, (4) students misinformed concerning high school requirements for 
college entry, arid (5) students misinformed concerning relationship of 
specific high school courses to college success. The other type of evaluative 
criteria-'that relating to what causes quality of program--m1ght Include such 
Items as (1) the ratio of students to counselors, (2) the counselors 
assigned noncounselor duties, (3) the proper education and training of 
counselors, and (4‘ the use of advisory or consulting committees, etc. 



We must have research which will establish the validity of our intuitive 
identification of those factors which cause quality of programs. In other 
words, we must establish a connecting link between those items which show 
quality of program and those items which cause the quality of program. Only 
those items which show definite linkage can we accurately say are important 
as we evaluate guidance programs and attempt to determine the procedures for 
improvement. 



Areas of Major Concern 

The earlier speaker reviewed for you the major concerns about counseling 
programs which surfaced in our review of the literature. It would appear that 
these concerns may not be universally held by all individuals in all states. 
Different groups may have different concerns between states and within 
states; and, there may be various degrees of concern related to different 
factors. How does one go about identifying the concerns which persist 
within his own sphere of operation? We would like to present to you a 
suggested procedure for determining these concerns and show you the results 
of having followed this procedure in Oklahoma. 

We felt that the people who should be able to identify the concerns 
of the state came more or less by groups. We first considered administrators-- 
the state director of vocational and technical education and his assistants, 
state supervisory personnel, and state level administrators In the State 
Department of Education. Also, we felt that we should ask others to look 
statewide at the concerns surrounding the program. These Individuals would 
Include local administrators, students, teacher- and counselor-educators, and 
others In education. Finally, we Identified other individuals such as 
members of the State Board for Vocational and Technical Education and the 
o 

ERIC 



22 



State Advisory Council who should be consulted. These are the types of 
people we feel should be included In any state survey designed to identify 
the major concerns relative to guidance and counseling programs. 

As we in Oklahoma identified what we considered to be major concerns, and 
as we developed procedures for validating this within a state, we could see 
the need for one more step--test1ng of the procedure. In order to accomplish 
this, representatives of the groups described above were presented with a 
list of concerns as gathered from the review of literature and were asked 
to rate these concerns on an 11-point continuum ranging from a major concern 
to no concern. Provisions were also made for the interviewee to Indicate 
that a concern which was not In the realm of the Influence of the counselor 
could be checked as not applicable. The result of this procedure gives us 
the ranking of the concerns which were indicated on the instrument. Individuals 
were also asked to add additional concerns which they considered of major 
Importance and many did respond this way. Following Is a ranking of the 
concerns which were of greatest Importance In the survey done In Oklahoma. 

Major Concerns in Oklahoma 

In response to the question, "What are the major concerns which you 
have about the present program of career guidance, counseling, and placement 
In the public schools of Oklahoma?" selected Individuals ranked the 
identified items In the following order. (The number score Is based on a 
maximum of 11 points). 
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Concerns and average ratings as follows: 



Concerns Average Ratin g 

The ratio of students to counselors 10.27 

Schools with no counselors 10.13 

Students, parents, and general public not adequately 
informed about dignity, challenge, and satisfaction 
(intellectual and financial) of a vocational- 
technical skilled job 10.11 

Failure to provide a saleable skill to graduates 

and school leavers 9.88 

Students not given experiences which prepare them 

to make wise educational and occupational choices 9.88 

Closer follow-up of students 9.66 

Lack of flexibility in curriculum to allow proper 

mix of academic and vocational choice 9.61 

Lack of comnunlcatlon between all agencies Involved 
In educational training and employment assistance 9.61 

Starting a comprehensive guidance program at 

beginning of formal education 9.61 

Lack of preparation of counselors In occupational 

areas 9.50 

Lack of relevance of education to career choice 

and advancement 9.50 

Counselors assigned noncounselor duties; l.e., 
discipline, attendance, study hall, etc. 9.38 

Lack of a systematic procedure which will enable 

guidance staff to acquire and expand knowledge about 

the world of work 9.33 

Overemphasis on Importance of baccul aureate degree 
to students 9.31 

Programs not designed to meet Individual needs, 

desires, or abilities 9.29 

Counseling programs which reflect only the atti- 
tudes and educational philosophy of a single 
school administrator 9.28 

Proper packaging and dissemination of world of 

work Information 9.05 



Continued research in career guidance 



8.83 



Danger of the development of two systems of 
guidance; i.e., academic and vocational 


8.72 


Misinformation given to students concerning high 
school requirements for college entry 


8.72 


Lack of use of comnunity resources in career 
exploration and guidance 


8.66 


The danger of separating programs for the disad- 
vantaged from regular vocational programs 


8.62 


Support personnel for counselors; I.e., test 
technicians, placement specialists, Information 
specialists, etc. 


8.58 


A systematic procedure for Implementation of new 
Ideas In counseling 


8.52 


Lack of functioning advisory committee 


8.38 


Need for large number of remedial programs at end 
of formal education 


8.11 


Lack of assistance from the Federal level to state 
and local guidance personnel 


7.70 


Guidance services set apart from rest of 
educational services 


7.58 

Mean 9.12 


Evidences Needed for Evaluation 




He, the team preparing this paper, felt that the persons who should be 


responsible for determining the basis upon which evaluation would be made 



should be those persons for whom the service is being performed. This 
means that we must ask state level administrators, regional administrators, 
local administrators and teachers, students, and parents, the kind of Informa- 
tion, the kinds of data that they will accept as evidence of quality In 
programs. If the administrator or the person for whom the service Is 
being performed will answer the question which we posed to administrators 
and others In Oklahoma, we can gather the data required In order to evaluate* 



We asked these individuals, "What evidence will you accept as Indicators of 
quality (high or low) in programs of career guidance, counseling, and 
placement? 1 ' From a search of the literature again, we were able to identify 
what we considered a list of the most often mentioned evidences needed for 
evaluation. This list compiled from our reading may not be appropriate 
for your particular state. We are suggesting a procedure for determining 
within your own state the major types of data and Information needed to 
evaluate programs. We are presenting to you the results of having moved 
through this procedure in a state to determine that state's requirements for 
evaluation. Probably the most difficult part of evaluation is to determine 
the kinds of data and information which should be gathered. After this 
the job of the evaluator becomes considerably simpler, but it must be the 
people for whom the service is being rendered that can give us the most 
accurate, the most acceptable answer as to the question of Identifying the 
criteria upon which we evaluate programs. 

Following is the order in which the evidences for evaluation were 
ranked by our respondents and the average rating on the 1 1 -point scale. 

Evidence Average Rating 

Extent to which post high school activities 

(Jobs, additional education, etc.) of graduates 

and school leavers are consistent with their 

high school r ■'gram 10.58 

Percent of high school graduates who have either 
a Job entry skill or plans for continuing education 10.27 

Extent to which students who are able, finish high 

school 10.22 

Percent of students dropping out of college the 

first year 10.16 
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Percent of counselor education dealing with occupa- 
tions requiring vocational -technical training 10.11 

Extent to which student has had experiences which 

help to develop his ability to make educational 

and/or vocational decisions 10.05 

Percent of counselor's time spent on activities 

other than counseling, guidance, and placement 10.05 

Ratio of counselors to students 9.88 

Percent of counselors who participated in in- 

service training for vocational counseling 9.77 

Percent of students who have current occupational 
Information easily accessible to them 9.77 

Percent of time spent by counselor on placement 

of students and/or school leavers 9.72 

Percent of schools having a written plan for 

guidance services and a job description for each 

counselor 9.58 

Percent of counselors having up-to-date experience 
in business and industry 9.55 

Number of areas of expertise represented on state 
staff; l.e., elementary, testing, vocational, 
research, evaluation, administration, secondary, 

Information, publications 9.52 

Percent of Jr. High School students who have 

participated In career exploration and testing 

for work aptitude 9.44 

Percent of counselors having In-service training 
In vocational education available to them 9.44 

Is this educational and/or vocational plan realistic 

In terms of students' test scores and/or grade 

point averages 9.44 

Percent of vocational student graduates who get a 

related Job or continue In related training 9.27 

knowledge of the extent to which guidance programs 

are Influencing the school curriculum at all levels 9.27 

The percent of students having an educational and/or 
vocational plan 8.94 
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Extent to which materials from state level are identi- 



fied and/or made available to local counselors 8.83 

Changes in enrollment in vocational programs in school 
or area school 8.75 

Percent of students of different levels of ability 
participating in vocational education programs 8.50 

Length of time required from request to visit by 

state staff 8.05 

State Staff - Number of visits per school year 7.29 

Kean 9.45 



Additional Areas of Concerns and Evidences for Evaluation 

Respondents to the rating scale administered In Oklahoma were given the 
opportunity to mention concerns and/or evidences which were not mentioned in 
the survey Instrument.,, Following are the Items which were suggested as being 
Important. 



Concerns 



1. Counselors given responsibility for discipline 

l . Lack of reliable Information available to counselors on 
quality o f programs of proprietary schools 

3. Separation of courses of study Into "prestigious" and "non- 
prestlglous" 

4. Financing so that counselors are not counted In calculating 
teacher-student ratio 



Evidences 



1. Completeness of Information kept on students 

2. Graduates reaction to quality of counseling and subsequent 
services 

3. Background - Past experiences of counselors 




28 



4. Training In how to work with students having special 
needs (handicapped, disadvantaged) 

5. Evidence of communication between counselors, teachers, and 
administrators 

6. Involvement of parents In counseling 

7. Amount of funds spent on career exploration materials 

With this list of acceptable evaluative criteria put In rank order, It 
Is possible for those of us responsible for evaluation to go as far down the 
list as our resources will allow. We will not In most Instances be able to 
answer all the questions presented by the data requests, but we should be 
able, with the help of the procedure outlined, to Investigate guidance programs 
in the light of what is considered most essential to realistic evaluation. 

We must constantly keep in mind that the criteria we set for evaluating 
programs— if that criterion is known by those to be evaluated— causes program 
emphasis to be placed on those criteria and program operation to move in the 
direction specified by the criteria. Thus if we establish as acceptable 
evidence, for example, "percent of high school graduates who have either 
a job entry skill or plans for continuing education," and if this criteria 
is embraced by administrators (state and local) and made known to counselors, 
we could expect to see a decrease in the percentage of students finishing 
without either a saleable skill or plans for further education or training. 

Of course, the establishment of a system for rewarding those who excel in 
reaching higher degrees of accomplishment of the established objectives will 
further exhilarate the change process. 

If administration has not really accepted the evaluative factors or if 
practitioners think administrators are only paying "lip service" to the 
criteria or if operational personnel are unaware of the criteria, very little 
is likely to happen in this direction. We must be very careful in the selection 
of the items upon which programs will be judged and we must be active in expressing 
and supporting the evaluative items wh1ct£t$ve been established. 



